Method of manufacturing a solid state image pickup apparatus having multiple transmission paths

ABSTRACT

An arithmetic circuit, which is retained by each pixel in a conventional image sensor, is shared by each column. Signal processing circuits of different configurations are provided on signal transmission paths in an upward direction and a downward direction of a vertical signal line for extracting an image signal from each pixel, whereby image output processing and arithmetic processing are performed completely separately by the different circuit blocks. Thus, image quality of an actual image is improved and optimum design for arithmetic processing is made possible. Specifically, an I-V converter circuit unit, a CDS circuit unit and the like are provided on the image output side. A current mirror circuit unit, an analog memory array unit, a comparator unit, a bias circuit unit, a data latch unit, an output data bus unit and the like are provided on the arithmetic processing side.

The subject matter of application Ser. No. 10/245,966 is incorporatedherein by reference. The present application is a continuation of U.S.application Ser. No. 10/245,966, filed Sep. 18, 2002 now U.S. Pat. No.6,858,827, which claims priority to Japanese Patent Application No.JP2001-287625 filed Sep. 20, 2001, and JP2002-169577, filed Jun. 11,2002. The present application claims priority to these previously filedapplications.

BACKGROUND OF THE INVENTION

The present invention relates to a solid-state image pickup apparatusand a control method thereof having a function of obtaining a normalimage signal and additionally a computational function for executingvarious applications.

Recently, solid-state image pickup apparatus have been proposed whichimage sensors have a function of performing various operations on imageinformation and thereby realize an increase in the speed of imageprocessing and the like.

As one of such image sensors, a sensor having a function of obtaining anormal actual image, a function of three-dimensional range calculation,and a function of detecting a moving object has been proposed (seeISSCC/2001/SESSION6/CMOS IMAGE SENSORS WITH EMBEDDED PROCESSORS/6.4(2001IEEE International Solid-State Circuits Conference) and JapanesePatent Application No. 2000-107723, for example; in the followingdescription, the former will be described as first conventionalliterature, and the latter will be described as second conventionalliterature).

The image sensor has therein a circuit for obtaining an image, whichcircuit is the same as in an ordinary image sensor, and additionally afunction of detecting temporal change in intensity of light. As aconcrete architecture, an image sensor in which each pixel has thecomputational function has already been reported.

Various image processing is realized by using the computational functionof such an image sensor. Principles of three-dimensional measurement,which is typical of the various image processing operations, will bedescribed in the following.

FIGS. 6A, 6B, and 6C are diagrams of assistance in explaining principlesof a triangulation method and a light stripe detecting method forthree-dimensional measurement.

As shown in FIG. 6A, in the triangulation method, a sensor 2 and a lightsource 3 are disposed at a distance from an object 1 of rangemeasurement. The light source 3 is intended to irradiate the object 1with a stripe of light, and is provided with a scanner (scanning mirror)4 for reflecting the stripe of light.

With such an arrangement, the scanner 4 repeats an operation of scanningthe stripe of light of the light source 3 from the right to the left,for example.

As shown in FIG. 6B, while the stripe of light of the light source 3 isscanned from the right to the left once, a few thousand to a few tenthousand frames are scanned in the sensor 2. Each imaging pixel in thesensor 2 outputs data indicating that the stripe of light reflected bythe object 1 is detected, at the time of the detection.

Directing attention to one pixel, a distance between the object 1 andthe sensor 2 in a direction of a line of sight of the pixel and a swingangle of the scanner 4 at the time of the detection of the reflectedlight are uniquely determined. Specifically, when scanning of thescanner 4 and counting the number of frames of the sensor 2 are startedat the same time, the swing angle of the scanner 4 is determined byknowing a frame count at which the stripe of light is detected, wherebythe distance between the object 1 and the sensor 2 is determined.

In an actual image sensor, the frame count and distance are corrected inadvance by an object for distance correction, and resulting data isretained as a table on the system side. Thus, high-precision absolutemeasurement of distance is made possible.

A function required of the image sensor in the triangulation method asdescribed above is high-sensitivity detection of the stripe of light.

Infrared light is generally used for wavelength of the light. However,reflectance of the infrared light differs depending on the measuredobject. Thus, for measurement of even an object having a texture of lowreflectance, accuracy in detecting passage of the stripe of light needsto be increased.

In the first conventional literature, light signal computation isperformed in each pixel for the high-sensitivity detection. Anarchitecture for this will be described in the following.

FIG. 7 is a block diagram showing a general configuration of an imagesensor in the first and second conventional literature. FIG. 8 is acircuit diagram showing an internal configuration of one pixel in theimage sensor shown in FIG. 7.

In FIG. 7, an imaging unit 10 for imaging a subject is provided thereinwith a large number of pixels 11 each forming a photosensor which pixelsare disposed in a matrix manner, vertical signal lines 12 for selectingeach of the pixels 11 and extracting an imaging signal from each column,and the like.

The imaging unit 10 has exteriorly thereof: a V scanner unit 13 forscanning the pixels 11 for extracting imaging signals in a verticaldirection through selecting lines; a signal generating unit 14 foroutputting a control signal to the V scanner unit 13; and output circuitunits 15 for receiving output signals of columns #1 to #192 from thevertical signal lines 12, performing necessary signal processing, andoutputting the result as image signals of the columns.

In FIG. 8, each of the pixels 11 includes: a photodiode (PD) 21 forreceiving light; an amplifying transistor (QA) 22 for passing a currentaccording to intensity of the light; a current mirror circuit 23 foramplifying the current; a current copier circuit (frame memories) 24 forstoring the current signal; a two-step chopper comparator 26 forcomparing currents from the current copier circuit 24 with each other;and a bias circuit (offset generator) 27 for applying a bias to thecurrents.

A unit for reading signal charge from the PD 21 in the pixel 11includes: a floating diffusion (FD) part 31 for extracting the signalcharge from the PD 21; a transfer transistor 32 for transferring thesignal charge from the PD 21 to the FD part 31; a reset transistor 34for resetting the FD part 31; the above-mentioned amplifying transistor22 for converting the signal charge from the FD part 31 into a voltagesignal and amplifying the voltage signal; the above-mentioned currentmirror circuit 23 for amplifying an output current of the amplifyingtransistor 22; and a switch (SA) 33 for controlling output timing.

The current copier circuit 24 has four circuits (M1 to M4) set inparallel with each other. The circuits each function as a frame memory,and are capable of storing light signals for a total of four frames.

FIG. 9 is a timing chart of range measurement operation by the imagesensor.

During one scan period in which the laser scans once, operation for afew thousand to a few ten thousand frames is performed in the imagesensor. One scan period in this case is generally adjusted to a videoframe rate when range information is imaged on the monitor, and is about33 msec.

In the following description, to be differentiated from the video framerate, a scan of one frame within the image sensor will be referred to asa sensor frame.

At a start of a sensor frame (1 frame in FIG. 9), a reset signal (RST)and a charge transfer signal (TX) of the FD part 31 in each pixel causea charge accumulated by a light signal to be transferred to the FD part31, that is, a gate of the amplifying transistor (QA) 22.

Thereafter, pixels on each line in a row direction of the image sensorare selected, and an operation of storing a signal in the current copiercircuit 24 (φ1) and reading operations (φ2 and φ3) are performed.

In the storing operation φ1, a detection signal is stored in one framememory. The storing frame memory is changed sequentially with eachchange of frames (frame index: A, B, C, D).

In the reading operations φ2 and φ3, memories of first two frames andmemories of second two frames are each added together, and then comparedby the comparator 26, whereby the following operation is made possible:f(k)+f(k−1)−(f(k−2)+f(k−3))−(Iz−Ic)  (Equation 1)

where the last Iz and Ic refer to bias currents of the bias circuit 27and correspond to currents in the periods φ2 and φ3, respectively, with(Iz−Ic)>0 in normal settings.

When no stripe of light is detected, no temporal difference occurs inintensity of light detected in each pixel, and therefore a calculationup to the fourth term of the equation 1 is zero. Thus, only the biasportion is left to provide a negative value. The negative value isoutputted as low data by the comparator 26.

When a stripe of light passes the pixel, on the other hand, there alwaysoccurs a time region where data of an addition of first two framesbecomes greater than data of an addition of second two frames (FIG. 6C).When a difference between the data of the addition of the first twoframes and the data of the addition of the second two frames exceeds theset bias (Iz−Ic), the operation of the equation 1 results in a positivevalue. Thus, directing attention to a certain pixel, the comparator 26outputs low data during a normal time, and outputs high data when astripe of light passes.

Therefore, when a frame count at which the comparator 26 outputs highdata is recorded for each pixel on the system side, a distance to eachpoint can be uniquely measured by triangulation from a relation of thecount and the angle of the light scanner.

The image sensor described above can also output a normal image byperforming A/D conversion processing within the pixels.

In this case, a reference signal is stored in one of the frame memoriesM1 to M4. Then, each time a sensor frame is scanned, a light signalcharge is accumulated by integration in the FD part 31 in the pixel,then stored in the other of the frame memories M1 to M4, and compared inmagnitude with the reference level by the comparator 26.

The reference level is exceeded by charge accumulation by a small numberof frame scans when the pixel has a high light intensity, whereas alarge number of frame operations are required when the pixel has a lowlight intensity. Thus, as in range measurement, when a frame count atwhich the data of the comparator 26 is inverted is stored on the systemside, the frame count corresponds to an actual image, which can be thenshown on the monitor.

However, since the image sensor as described above retains thecomputational circuit in each of the pixels, the pixels have a largesize, and therefore it is difficult to reduce size of the sensor andincrease the number of pixels of the sensor.

In addition, the large circuit scale results in a high power consumptionby the chip, specifically a power consumption of 1 W or more accordingto the first conventional literature, for example.

Such an image sensor is usable in a relatively large system with asufficient disposing space and high power capacity, but is not suitablefor consumer applications and the like requiring a reduction in powerconsumption, a reduction in cost, and an increase in the number ofpixels of an actual image.

Furthermore, as in the case of an ordinary imager, there is a tendencyto require a high-quality color image as the actual image in consumerapplications.

SUMMARY OF THE INVENTION

It is accordingly an object of the present invention to provide asolid-state image pickup apparatus and a control method that can enhanceboth a function of obtaining a normal actual image and a computationalfunction for executing various applications without complicating aconfiguration within each pixel and thereby achieve a reduction in size,power consumption, and cost, an increase in the number of pixels (higherpicture quality) of an actual image and the like.

In order to achieve the above object, according to the presentinvention, there is provided a solid-state image pickup apparatusincluding: a plurality of light receiving units each forming an imagingpixel; a plurality of photoelectric conversion units for convertinglight received by the light receiving units into electric signals; asignal line having a plurality of signal transmission directions forextracting the electric signals converted by the plurality ofphotoelectric conversion units; a first signal processing unit forprocessing the electric signals transmitted in a first signaltransmitting direction through the signal line; and a second signalprocessing unit for processing the electric signals transmitted in asecond signal transmitting direction through the signal line; whereinthe signal processing performed by the first signal processing unit andthe signal processing performed by the second signal processing unit aredifferent from each other.

With the solid-state image pickup apparatus according to the presentinvention, the image signals obtained by the imaging pixels aretransmitted in the first signal transmitting direction and the secondsignal transmitting direction through the signal line, and the firstsignal processing unit and the second signal processing unit performsignal processing operations different from each other. Thus, normalimage signal output and various other arithmetic processing, forexample, can be performed by separate circuits.

Therefore, circuit elements required for each of the signal processingoperations are arranged together outside pixels, so that a configurationwithin each of the pixels can be simplified and minimized. Also, it ispossible to enhance the function of obtaining a normal actual image andthe computational function for executing various applications by theirrespective independent circuit configurations. It is thus possible toachieve smaller apparatus size, lower power consumption, lower cost, anincrease in the number of pixels of an actual image (higher imagequality) and the like.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a plan view of a general configuration of an image sensoraccording to an embodiment of the present invention;

FIG. 2 is a block diagram showing an internal configuration of each ofpixels of a pixel array unit in the image sensor shown in FIG. 1;

FIG. 3 is a block diagram showing details of each of blocks of the imagesensor shown in FIG. 1;

FIG. 4 is a timing chart of operation at the time of image output by theimage sensor shown in FIG. 1;

FIG. 5 is a timing chart of operation at the time of range measurementby the image sensor shown in FIG. 1;

FIGS. 6A, 6B, and 6C are diagrams of assistance in explaining principlesof a triangulation method and a light stripe detecting method forthree-dimensional range measurement;

FIG. 7 is a block diagram showing a general configuration of aconventional image sensor;

FIG. 8 is a circuit diagram showing an internal configuration of a pixelin the image sensor shown in FIG. 7;

FIG. 9 is a timing chart of range measurement operation by the imagesensor shown in FIG. 7;

FIG. 10 is a perspective view of an example of structure of a MEMSmirror used in a three-dimensional range measurement system according toa second embodiment of the present invention; and

FIGS. 11A and 11B are diagrams of assistance in explaining configurationexamples of a three-dimensional range measurement system according to athird embodiment of the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

A preferred embodiment of the present invention will next be describedwith reference to the drawings.

In a solid-state image pickup apparatus (hereinafter referred to as animage sensor or simply as a sensor) according to the present embodiment,an arithmetic circuit, which is retained by each pixel in the imagesensor described in the conventional example, is shared by each column.The solid-state image pickup apparatus performs image output processingand arithmetic processing completely separately by different circuitblocks (a first signal processing unit and a second signal processingunit). The solid-state image pickup apparatus thereby achieves highimage quality of an actual image and enables optimum design forarithmetic processing.

The image output processing and the arithmetic processing are selectedby a selection signal from an exterior of the sensor. When one of theimage output processing and the arithmetic processing is not performed,the sensor effects control to stop operation of part or the whole of thecircuit block.

A method of scanning image outputs of the sensor and a method ofarithmetic operation in image processing (applied to range measurementas an example, as in the conventional example) according to a concreteembodiment of the present invention will hereinafter be described.

FIG. 1 is a plan view of a general configuration of an image sensoraccording to a first embodiment of the present invention.

As shown in FIG. 1, the image sensor includes a pixel array unit 41, apixel V scanner unit 42, a pixel H scanner unit 43, an I-V convertercircuit unit 44, a CDS circuit unit 45, a current mirror circuit unit46, an analog memory array unit 47, a memory V scanner unit 48, a memoryH scanner unit 49, a comparator unit 50, a bias circuit unit 51, a datalatch unit 52, an output data bus unit 53 and the like.

The pixel array unit 41 is formed by arranging a plurality of pixels 411for detecting light in a two-dimensional matrix form in a row directionand a column direction. A signal sent out from each of the pixels 411 istransmitted by a signal line (vertical signal line) 54 extending in avertical direction.

The pixel V scanner unit 42 and the pixel H scanner unit 43 scan aninterior of the pixel array unit 41 in the vertical direction and thehorizontal direction to thereby select one of the pixels 411.

The I-V converter circuit unit 44 converts a current outputted from eachof the pixels 411 to a horizontal signal line 55 at the time of normalimage output scanning into a voltage signal. The CDS circuit unit 45subjects the output signal from the I-V converter circuit unit 44 topredetermined noise rejection processing, and then outputs the result asan image signal.

The current mirror circuit unit 46 amplifies the output current fromeach of the pixels 411 at the time of arithmetic processing. The analogmemory array unit 47 temporarily stores an output from the currentmirror circuit unit 46 within a current copier cell.

The memory V scanner unit 48 and the memory H scanner unit 49 scan thecurrent copier cell in the analog memory array unit 47 to extract datawithin the cell. The bias circuit unit 51 supplies a bias current to thecurrent copier cell in the analog memory array unit 47.

The comparator unit 50 subjects the data read from the analog memoryarray unit 47 to a comparing operation. The data latch unit 52 latchesoperation data of the comparator unit 50, and outputs the latched datafrom the output data bus unit 53.

The vertical signal line 54 is provided with a switch SW1 for making andbreaking connection between the pixel array unit 41 and the horizontalsignal line 55, a switch SW2 for making and breaking connection betweenthe pixel array unit 41 and the current mirror circuit unit 46, a switchSW3 for making and breaking connection between the current mirrorcircuit unit 46 and the analog memory array unit 47, and a switch SW4for making and breaking connection between the analog memory array unit47 and the bias circuit unit 51.

With such a configuration, at the time of normal image output scanning,the pixels 411 of the pixel array unit 41 are sequentially scanned bythe pixel H scanner unit and the pixel V scanner unit to select oneparticular pixel 411. A current signal sent out from the pixel 411 istransmitted in an upward direction (first signal transmitting direction)in the figure. The pixel H scanner unit 43 sequentially selects theswitch SW1. A signal from each pixel is thus transferred to thehorizontal signal line. The signal is thereafter converted by the I-Vconverter circuit unit 44 into a voltage signal. Further, FPN (FixedPattern Noise) and reset noise (ktc noise) are removed from the voltagesignal by the CDS circuit unit 45, and then the result is outputted asan image signal output.

Basically, a scanning procedure at the time of normal image output asdescribed above is conventionally known (see ISSCC/2000/SESSION6/CMOSIMAGE SENSORS WITH EMBEDDED PROCESSORS/6.1 (2000IEEE InternationalSolid-State Circuits Conference), for example). Therefore detaileddescription of the scanning procedure will be omitted.

At the time of such image output scanning, the switch SW2 of thevertical signal line 54 is turned off, whereby the pixel array unit 41is cut off from the current mirror circuit unit 46.

On the other hand, at the time of range measurement, the switch SW1 isturned off, and the switch SW2 and the switch SW3 are turned on. Thecurrent signal is therefore transmitted to the current mirror circuitunit 46 disposed in a downward direction (second signal transmittingdirection) in the figure of the vertical signal line 54.

At this time, the pixel V scanner unit 42 simultaneously selects allpixels in the same row of the pixel array unit 41, and therefore asignal from each column is simultaneously outputted in parallel (thatis, column-parallel operation is performed).

The signal transmitted to the current mirror circuit unit 46 isthereafter retained within the analog memory array unit 47. Thereafter,data of each frame is compared by the comparator unit 50. A result ofthe comparison is latched by the data latch unit 52 and then outputtedfrom the output data bus unit 53.

FIG. 2 is a block diagram showing an internal configuration of each ofthe pixels 411 forming the pixel array unit 41. FIG. 2 shows four pixelscorresponding to four complementary colors Y, G, Cy, and Mg.

Each of the pixels 411 is formed with a photodiode (PD) 60 and five MOStransistors 61 to 65.

Light received by the PD 60 is converted into a charge, and the chargeis transferred by a transfer transistor 61 to a floating diffusion (FD)part 66. The charge transferred to the FD part 66 determines gatepotential of an amplifying transistor 64. A current corresponding to thegate potential passes through the amplifying transistor 64 and aselecting transistor 65, and is then transmitted to a vertical signalline (SIG_n) 54.

A reset transistor 63 is provided to reset the FD part 66 to a powersupply voltage. A transfer selecting transistor 62 is provided to selecta gate of the transfer transistor 61.

Signal timing at the time of image output and at the time of rangemeasurement will be described in the following.

FIG. 3 is a block diagram showing details of each of the blocks shown inFIG. 1. FIG. 4 is a timing chart of operation at the time of imageoutput. FIG. 5 is a timing chart of operation at the time of rangemeasurement.

The operation at the time of image output will first be described withreference to FIG. 4.

Each switch SW2 in a lower portion of the vertical signal line 54 isturned off by a TSSW signal (low; inactive), so that the signal path iscut off.

First, when the pixel V scanner unit 42 selects a specific row, a one-Hperiod select line (SEL_n) becomes high (active), whereby the selectingtransistor 65 is turned on.

Then, a reset pulse is applied to a reset line (TDR_n−1), and the resettransistor 63 resets the FD part 66 to the power supply voltage. At thistime, a signal in the reset state is sent out to a signal line (SIG_n).

A transfer gate selecting line (TRG_n) connected to a gate of thetransfer selecting transistor 62 becomes high (active) in synchronismwith the pixel H scanner unit 43, and at the same time, a transferselecting line (TDR_n) connected to a drain of the transfer selectingtransistor 62 becomes high (active). Thus, the transfer transistor 61only in the specific pixel is selectively turned on, and a charge in thePD 60 is transferred to the FD part 66.

After completion of the transfer in each pixel, the transfer selectingline (TDR_n) is made low (inactive), and thereafter the transfer gateselecting line (TRG_n) is made low (inactive).

As a result of such an operation, a signal current corresponding tolight intensity is sent out to the signal line (SIG_n). The signalcurrent is passed through the horizontal signal line 55 and convertedinto a voltage signal by the I-V converter circuit unit 44, and thennoise rejection is performed from the voltage signal by the CDS circuitunit 45.

The reset signal and the image signal are sequentially transferred tothe CDS circuit unit 45. The reset signal and the image signal areclamped by an SHR pulse and an SHD pulse to perform noise rejection.

The operation at the time of range measurement will next be describedwith reference to FIG. 5.

The three-dimensional measurement is similar to that of the conventionalexample in that triangulation is performed by detecting passage of alight stripe in each pixel.

At the time of this scanning, unlike the time of image measurement (thatis, image output), the switch SW1 in an upper portion of the verticalsignal line 54 is turned off, whereby the signal line SIG isdisconnected from the horizontal signal line 55.

A video frame period (1 scan period) and a sensor frame period (1 frame)are similar to those in the foregoing conventional example, and aone-line period (1 row access period) is different from that in theconventional example.

FIG. 5 shows scanning in the one-line period. Scanning in the one-Hperiod will be described in the following.

In scanning at the time of range measurement, four pixels constitutingcolor filters (for example Y, G, Cy, and Mg in a case of a complementarycolor mosaic filter) within the pixel array unit 41 are regarded as onelight receiving pixel unit, and signals from the four pixels are added(merged) together for processing. That is, the signals of the fourpixels are handled as a signal of one pixel.

As shown below, this is to increase sensitivity to compensate anoperating frame rate at the time of range measurement about 100 timesfaster than at the time of image measurement and hence a shorter lightreceiving time in each pixel, and also to compensate difference oftransmittance of the filters when receiving reflected infrared stripelight at the time of range measurement.

A unit in which the four pixels are regarded as one pixel willhereinafter be described as an ROU (Range Operating Unit) (FIG. 3).

Since the range measurement is column-parallel operation in whichreading in all columns is performed at a time, all transfer gateselecting lines TRG in the pixel array unit 41 remain high (active) atall times.

Unlike the time of image measurement, for addition (merging) of thesignals from the four pixels, the pixel V scanner unit 42 scans so as toselect two rows at the same time.

Specifically, in FIG. 3, SEL_n and SEL_n+1 are selected at the sametime. By selecting two specific rows, all pixel selecting transistors 65in the two rows are turned on.

Thereafter, a pulse is applied sequentially to TDR_n−1, TDR_n, andTDR_n+1.

A TDR line is connected to both a drain of a transfer selectingtransistor 62 in a pixel and a gate of a reset transistor 63 on a nextline. With the TDR line, charge is transferred from a PD 60, and at thesame time, resetting is performed on the next line. That is, in thisexample, the TDR line is shared by the transfer selecting line and thereset line.

Thus, by first applying a pulse to TDR_n−1, pixels in an nth row arereset. By applying a pulse to a TDR_n line, charge in the pixels in thenth row is transferred, and at the same time, pixels in an (n+1)th roware reset.

By applying a pulse to TDR_n+1, charge in the pixels in the (n+1)th rowis transferred. At the same time, pixels in a next (n+2)th row arereset, which has no particular significance in the series of scans (thatis, which produces no effects).

Thus, in this example, when reading an image by simultaneously selectinga plurality of rows, the select lines (SEL_n and SEL_n+1) in theplurality of rows are activated, and thereafter an active pulse issequentially applied to the selecting lines (TDR_n−1, TDR_n, andTDR_n+1) shared by transfer selecting lines and reset lines inincreasing order of selected row number.

As a result of the above scanning, charges generated by receiving lightare simultaneously transferred to the FD parts 66 of the pixels in thetwo rows, and signal currents from the amplifying transistors 64 of twopixels simultaneously flow into each signal line. Thus, signal additionfor two pixels is performed first.

A resulting current flows into a current mirror circuit 461simultaneously with the turning on of the CMOS switch SW2 in the lowerportion of the signal line. At this time, an odd-numbered column and aneven-numbered column are connected to each other in front of the currentmirror circuit 461, whereby currents from the two columns flow into thesingle current mirror circuit 461.

In addition to the simultaneous selection of the two rows, signaladdition for four pixels is thereby completed. A resulting current isamplified by the current mirror circuit 461, whereby a currentcorresponding to the signals is led in through a signal line SIM_(n+1)/2within the memory array 47.

The analog memory array unit 47 formed by current copier cells has fourcurrent copier cells 471 corresponding to one foregoing ROU. The fourcurrent copier cells correspond to memories for four sensor frames.

The memory array unit 47 is scanned by the memory V scanner unit 48 insynchronism with the scanning of the pixel array unit 41.

The current copier cells for four frames are arranged in the same manneras the pixel ROU. Therefore, when two rows of the pixel array unit 41are simultaneously selected, the memory V scanner unit 48 simultaneouslyselects two rows, which corresponds to selecting a unit of currentcopier cells for four frames.

This unit of the memory array unit 47 will hereinafter be described asan RMU (Range Memory Unit) (FIG. 3).

At the same time the switches SW2 and SW3 are turned on by the TSSWsignal, one line selecting line CCS_n1 is driven high, and thus aselecting transistor 73 of one frame memory in the RMU is turned on.

A current flows from a PMOS transistor 71 forming the memory cell to thecurrent mirror circuit unit 46. At this time, a memory switch transistor72 within the cell is turned on, and thus the PMOS transistor 71 has adrain and a gate biased to the same potential. Hence, the gate potentialis maintained at a potential that is determined according to the currentflowing in the cell.

After a steady state is reached in such conditions, a CCM_n1 signalturns off the transistor 72 to isolate the gate and the drain of thetransistor 71 from each other, whereby the above gate potential isdynamically stored and retained by the gate of the transistor 71.

The above signal storing phase will be denoted by φ1.

In next timing, signal comparison between frames is performed to detectpassage of a light stripe.

As in the conventional example, the signal comparison between framessubjects signal additions of two temporally preceding frames and twotemporally succeeding frames to subtraction, and thus the operation ofthe (equation 1) is carried out.

In this reading operation, the switches SW2 and SW3 are turned off, andthe switch SW4 under the memory array is turned on by a BISW signal. Atthe same time, selecting lines of two frames to be read in the RMU aredriven high.

Thus, current signals read from the memory cells flow into a loadtransistor 74, and potential of a memory signal line (SGM) is stabilizedto a potential determined by current capacity of the load transistor 74and the PMOS transistors 71 of the memory cells (for two frames).

In order to carry out the (equation 1), memory cell selecting linesCCS_n3 and CCS_n4, for example, are selected simultaneously, whereby thetwo preceding frames are read simultaneously. Thus, a resulting signalpotential is inputted to an input gate of a chopper comparator connectedto the vertical signal line 54. Transistor initializing switches TC1 andTC2 of the two-step chopper comparator are sequentially turned on, andthe comparator is initialized by the signal potential of the twopreceding frames.

Next, in order to read signals of the second two frames, selecting linesCCS_n1 and CCS_n2 are selected simultaneously, currents are sent to thesignal line as in the case of the two preceding frames, and a signalpotential is determined.

At this time, since the comparator is initialized by the two precedingframes, when the signal potential is greater than that of the twopreceding frames, an output of the comparator is high, whereas when thesignal potential is not greater than that of the two preceding frames,the comparator outputs a low. Thereby, the (equation 1) is carried out,enabling light detection.

The reading of the first two frames and the reading of the second twoframes are shown in φ2 and φ3 in the timing chart.

In this case, in order to weight the signals, bias current is suppliedfrom the bias circuit unit 51 in each of φ2 and φ3. The bias current isthe same as Iz and Ic in the conventional example, and makes it possibleto arbitrarily weight the signals from the analog memory array unit 47.As an example, the bias circuit unit 51 uses a source-follower circuitof a PMOS transistor biased by a column-shared current mirror.

In other than the reading operation, the switch SW4 provided between thebias circuit unit 51 and the analog memory array unit 47 is turned off,whereby the bias circuit unit 51 biases the potential of the signal linein a stage preceding the comparator to the power supply voltage.

A selecting signal of each of the switches SW2, SW3, and SW4 and CCMselecting signals in the memory array are subjected to a logical ANDwith an ADRn signal running vertically in FIG. 3.

When the operation in φ1 to φ3 is performed simultaneously in all thecolumns, there may be an increase in power consumption. Depending on thedegree of the increase, proper operation may not be performed as aresult of increase in a load on wiring within the elements.

In order to cope with such a situation, the circuit blocks below the SW2are divided (resulting units will be referred to as divided areas), andthe divided areas are scanned serially, so that a concentrated increasein power consumption is prevented.

In the timing chart of FIG. 5, a case where the whole of the circuitblocks is divided into four areas is assumed. A divided area is selectedby the ADRn signal, and thus the operation in φ1 to φ3 is performed ineach of the divided areas 1 to 4.

As described above, in this example, when reading signals from the pixelarray 41 in column parallel and performing arithmetic processing in eachcolumn, arithmetic areas are formed by a plurality of columns, and thearithmetic processing is performed serially in each of the arithmeticareas. Thus, an appropriate arithmetic operation in which loads aretaken into consideration is possible.

Each of the arithmetic areas is selected by the address line. Theswitches in stages preceding and succeeding the current mirror unit, amemory switch line, a switch in a stage succeeding the comparator, aline for selecting the load transistor of the signal line, a comparatorinitializing line, and an enable line of the data latch are operated bythe address line only at a time when the area is selected.

The two foregoing operations of the image measurement (image output) andthe range measurement can be performed independently of each other andcontinuously in different time periods, or may be alternated atintervals of an arbitrary number of video frames such as one videoframe.

In this case, the image sensor alternately outputs image information andrange information, thus enabling real-time image processing thatcombines the image information and the range information.

The above operations can be selected as required by an externallyinputted mode selecting signal MSL, for example.

The image sensor in this example can perform various other imageprocessing operations than the range measurement by circuit architecturefor the range measurement.

An example of the image processing operations is motion detection.

This is a function of extracting only a moving object within an imagingscreen by performing basically the same operation as the rangemeasurement. In the range measurement a difference between temporallysuccessive frames is calculated at all times. Thus, when there is amoving object in the image, as in detection of a light stripe, timing inwhich signal strength of the latter frames exceeds signal strength ofthe first frames occurs in the inter-frame difference operation of thecomparator.

Thus, detection of the timing enables detection of the moving object.

In the case of the motion detection, high-speed scanning of sensorframes as in the range measurement is not required; in view of detectionsensitivity, the frame rate can be reduced to a video frame rate at thelowest level.

In addition, with the above-described circuit configuration, imageinformation can be outputted in a digital form. A method for this isbasically the same as the image output method of the conventionalexample, and uses the analog memory array unit 47 and the comparatorunit 50.

While at the time of range measurement in the operation exampledescribed above, two lines are read simultaneously for four-pixeladdition, lines are scanned one by one in the digital image output usingthe analog memory array unit 47 and the comparator unit 50.

Also, rather than turning on the odd-numbered column and theeven-numbered column simultaneously, the switch SW2 separately turns onthe odd-numbered column and the even-numbered column by a SEL_ODD signaland a SEL_EVEN signal, respectively. That is, the reading of theodd-numbered column and the reading of the even-numbered column areperformed separately in a one-H period.

In this case, in the operation of reading each column, two framememories can be made to correspond to one pixel. Hence, when one framememory is used to retain a reference signal and the other frame memoryis used to retain an image signal, the reference signal and the imagesignal can be successively compared with each other by the integralcharge accumulation of the conventional example to thereby provide animage signal.

As another method for image output, in the one-H period, the referencesignal is read to initialize the comparator unit 50 as in the above,thereafter the image signal is read and sent to the comparator unit 50,and then the bias current is ramped, so that a level at which digitaldata is inverted is detected and thus image information can beextracted.

Furthermore, detection of an edge of an image is also made possible bycircuit modification.

Instead of only one column for one comparator input as described above,by enabling an input from an adjacent column by switch selection, levelof a signal from an adjacent pixel can be compared.

Thus, only a part of great signal change in image information can beextracted, that is, an edge can be detected.

In addition, various filter processing and smoothing processing usingfilters can be performed.

As described above, in the image sensor in this example, the arithmeticcircuit, which is retained by each pixel in the image sensor describedin the conventional example, is shared by each column. Also, the imagesensor performs image output processing and arithmetic processingcompletely separately by different circuit blocks. It is therebypossible to simplify configuration within each of the pixels, reducesize of the apparatus as a whole, achieve higher image quality of anactual image, and make optimum design for arithmetic processing.

For example, the order of scanning the pixels is changed between thenormal image output and arithmetic processing, so that optimumprocessing can be performed.

As to the order of scanning the pixels, serial processing in a unit ofone pixel or in a unit of a block of a small number of pixels andparallel processing using a plurality of signal lines can be properlyused. For example, serial processing in a unit of one pixel can beperformed at the time of image information output processing, whereasparallel processing using a plurality of signal lines can be performedat the other time of arithmetic processing. Thus, optimization to suitcharacteristics of each signal processing is possible.

In addition, the number of pixels transmitted simultaneously to a singlesignal line can be changed; at the time of image information outputprocessing, a signal of one pixel is transmitted to the signal line,whereas at the other time of arithmetic processing, signals of aplurality of pixels are transmitted simultaneously to the single signalline. Thus, optimization to suit characteristics of each signalprocessing becomes possible.

Also, a number of pixels corresponding to a combination of color filtersis used as the number of pixels transmitted simultaneously to the singlesignal line at the time of arithmetic processing. Thus, high-precisionarithmetic becomes possible.

In addition, in the above example, a TDR line is shared by a transferselecting line disposed in a certain row and a reset line disposed inthe next row. Thus, it becomes possible to reduce wiring space and sizeof the apparatus.

Moreover, in the above example, switches are provided each for one or aplurality of columns of the pixel array, a switch to be turned on at thetime of signal reading is selected from the switches, and therebycolumns for input to individual current mirror circuits are selected. Inaddition, a plurality of rows or a plurality of columns within the pixelarray are selected simultaneously to merge and add signals of aplurality of pixels together and thus handle the plurality of pixels asone light receiving pixel unit. It is thereby possible to employ amethod specific to the arithmetic processing. Thus, optimization ispossible.

Furthermore, in the above example, to compare a result of addition of acombination of two frames or more among a plurality of framescorresponding to one light receiving pixel unit in the analog memoryarray unit 47, memory cells corresponding to the frames are arranged ina matrix manner with a signal line in between, and cells arranged atopposite poles with the signal line in between are selected for thecombination of frames at all times. Also, selection of a plurality ofrows by the scanner of the pixel array 41 at the time of selecting onelight receiving pixel unit is synchronized with selection of a pluralityof rows by the scanner of the analog memory array unit 47 at the time ofselecting a unit of a plurality of frames.

Thus, high-precision and high-efficiency signal processing becomespossible.

A second embodiment of the present invention will next be described.

The solid-state image pickup apparatus according to the first embodimentas described above realizes functions of normal color image output andthree-dimensional range measurement based on a light-section method. Themethod of three-dimensional range measurement can be realized by theconventional configuration described with reference to FIG. 6A.

Specifically, a light source emitting a stripe of light and a scanningmirror are disposed in the vicinity of a sensor (light receiving unit).A subject is irradiated with the stripe of light via the mirror whilescanning the scanning mirror. Range information for each point of thesubject can be obtained from a relation between timing in which eachpixel of the sensor receives the stripe of light reflected from thesubject and a scan angle of the mirror on principles of triangulation.

In this case, however, many parts such as the light source, an opticaldevice for generating the stripe of light, the mirror, a driving systemfor scan operation and the like are required outside the sensor. It istherefore not easy to reduce size of the apparatus or save power. Acylindrical lens as the optical device for generating the stripe oflight would be required to satisfy optical conditions. It is thereforenot easy to reduce size of the lens.

Accordingly, in the second embodiment of the present invention, a mirrorunit performing scan operation in a system for three-dimensional rangemeasurement by the light-section method is formed by a MEMS (MicroElectro Mechanical System) mirror. Thus, a simple small system isrealized, and power consumption is reduced.

FIG. 10 is a perspective view of an example of structure of the MEMSmirror used in the three-dimensional range measurement system accordingto the second embodiment of the present invention.

The MEMS mirror shown in FIG. 10 is a scanner mirror of anelectromagnetic driving type formed on a silicon substrate (see “HiroshiMiyajima, Journal of Microelectromechanical Systems Vol. 10, No. 3 2001p418-p424,” for example).

The MEMS mirror is formed by attaching a moving plate (mirror body) 120having a mirror surface formed on a surface of the silicon substrate toa metal base 121. A fixed piece 120B is formed on both sides of themoving plate 120 with a hinge portion 120A in between. The fixed piece120B is joined to the metal base 121. Thus, using flexibility andelasticity of the hinge portions 120A, the moving plate 120 isrotatable.

In the metal base 121, a sensing coil 122 and a driving coil 123 aredisposed on a backside of the moving plate 120. Magnets 124 and a yoke125 are disposed so as to sandwich the moving plate 120.

A current is passed through the driving coil 123 via a flat cable 126 orthe like. A displacement of the moving plate 120 is detected by adetection signal of the sensing coil 122, and a value of the current tothe driving coil 123 is controlled. Thus, using a balance betweenLorentz force and torsional stress of the hinge portions, the movingplate (mirror surface) 120 is allowed to perform scan operation.

Incidentally, vibration frequency of the mirror, swing angle of themirror, and starting and stopping of the operation can be controlledexternally.

In addition, the example shown in FIG. 10 can select two driving modes:a galvanometric driving mode in which the swing angle of the mirrorsurface is controlled statically by adjusting the current in amount; anda resonant driving mode in which the vibrating operation of the mirrorsurface and an external control signal are synchronized with each otherto thereby cause resonant operation and hence the scan operation of themirror surface.

By forming the hinge portions 120A of the moving plate 120 of thinpolyimide films, it is possible for the MEMS mirror to performlow-frequency scan operation, which is difficult for an ordinary siliconhinge to achieve.

A conventional scanning mirror used in the light-section method hasproblems of large parts size and high power consumption because adriving unit such as a galvano-motor and a mirror are formed separately.In this example, as shown in FIG. 10, the system as a whole for thelight-section method can be reduced in size by using the MEMS mirror, inwhich the mirror and the driving unit are formed integrally with eachother.

It is to be noted that while this example uses a scanner of anelectromagnetic driving type as an example of the MEMS mirror, scanoperation using thermal expansion of hinge material by passing a currentthrough the mirror hinge portion, or scan operation using difference inthermal expansivity between layered films forming the hinge may beperformed as another driving method.

Furthermore, with a MEMS type mirror in which a mirror and a unit fordriving the mirror are formed integrally with each other on the samesubstrate, the system for the light-section method can be similarlyreduced in size.

Furthermore, since the MEMS mirror is generally formed on asemiconductor substrate, a light source for irradiating the scanningmirror such as a laser, an LED or the like can be formed integrally withthe MEMS mirror on the same substrate.

A third embodiment of the present invention will next be described.

FIGS. 11A and 11B are diagrams of assistance in explaining twoconfiguration examples of a three-dimensional range measurement systemaccording to the third embodiment of the present invention.

The system for the light-section method in this example uses a laserhologram 100 in a light projecting unit for providing a stripe of light.

The laser hologram is commercialized and used as an AF light source of adigital still camera, for example. As shown in FIGS. 11A and 11B, thelaser hologram is disposed in a path of light emitted from a laser lightsource 101, and controls laser light into a stripe of light and suppliesthe stripe of light to an object 102.

Light reflected from the object 102 is passed through a lens 111 andthen shot by a sensor 110 as described in the first embodiment, wherebythree-dimensional range measurement is performed.

It is to be noted that while in this example, the same mirror scanner(scanner mirror) 103 as in the example shown in FIG. 6A scans laserlight, the MEMS mirror shown in FIG. 10 can also be used.

By using the laser hologram 100, that is, by only adding a small opticalsystem (hologram element) to the laser light source 101, a light stripecan be generated. In addition, since the hologram element is formed byan inexpensive plastic substrate, the hologram element is advantageousalso in terms of cost.

The laser hologram 100 may be disposed between the laser light source101 and the mirror scanner 103, as shown in FIG. 11A, or may be disposedin a stage succeeding the mirror scanner, as shown in FIG. 11B.

When the laser hologram 100 is disposed in the stage succeeding themirror scanner 103, spotlight emitted from the laser light source 101 isscanned by the mirror, and then spread into a stripe.

As a fourth embodiment of the present invention for simplifying thelight source, the same sensor as in the first embodiment is used and alight source such as an LED or the like is used as the light source.Instead of scanning by the mirror, the fourth embodiment can beconfigured to calculate a difference between frames picked up when theLED is turned on and when the LED is turned off, so that an object inthe foreground against the background can be recognized.

Incidentally, with this configuration, rough range measurement can beperformed by effecting control such as changing intensity of light ofthe LED.

Examples of application of the foregoing embodiments will next bedescribed.

An imaging system formed by the solid-state image pickup apparatusaccording to each of the foregoing embodiments can be used to realizeimage processing, image recognition, and other functions that have beenconventionally difficult to realize with various IT apparatus and thelike as mentioned below.

FIRST APPLICATION EXAMPLE

In this example, the system for the light-section method according tothe foregoing first to fourth embodiments is used to effect control toextract an image in a certain distance range from an image on the basisof range information.

For example, this control is effected to cut out only an image of anobject at a shorter distance from the sensor and eliminate an imageportion in the background. This function makes it possible to extractonly an image of a person talking or a person being spoken to in theforeground in image communication on a portable telephone, a portableterminal (PDA), a videophone, a videoconference or the like. Thefunction can be used to conceal private information or the like bydeleting information such as a location of the person being spoken to orthe like. Furthermore, when an image to be transmitted is limited toonly a cut-out image, the function can be used to reduce an amount ofinformation of the image to be transmitted.

In addition, with the above function, a different background can be usedin place of the deleted background. Specifically, a landscape obtainedseparately or the like is used as the background, and thus thebackground can be changed according to personal preference. In thiscase, range information of the cut-out object and the backgroundfacilitates image superimposition.

The processing of extracting a particular image can be used aspreprocessing of image recognition and object recognition processing.For example, face recognition processing generally requires an operationof extracting an image of a face portion from the background aspreprocessing before an operation of recognizing the face portion. Sincean ordinary extracting operation uses only image information, theextracting operation has not been easy, taking a long processing timeand the like; however, the processing of cutting out the face portion orthe like can be readily performed by using the above system.

Incidentally, in this example, although the LED irradiation systemdescribed in the fourth embodiment can be used to realize a similarfunction, it is difficult for the LED irradiation system to effect finecontrol according to range information because of poorer range accuracyof the LED irradiation system.

SECOND APPLICATION EXAMPLE

In this example, two systems for the light-section method as shown inthe foregoing first to fourth embodiments are used to combine two imagesaccording to range information. Specifically, the two systems for thelight-section method as shown in the foregoing first to fourthembodiments are used to effect control to display an image in theforeground of two images at all times.

This for example makes it possible to virtually arrange an ornament,which is disposed at a different place, on a table in a room on thescreen as a virtual space simulation. Also, it is possible to obtain ona real-time basis an image in which a person virtually moves in a roomand hides behind an object, for example. These examples can be appliedto simulation of interior arrangements, interaction (interactive battletype) games and the like.

In addition, a user interface can be constructed by reflecting akeyboard, various buttons and the like and a manually generated image onthe screen.

Furthermore, some applications allow operation such as displaying animage in the background rather than displaying only an image in theforeground, and thus allow operation such as making visible a thingoriginally invisible.

In this example, the number of systems is not limited to two; three ormore systems can combine images. This example not only performsreal-time combining processing but is also capable of variations such ascontrol using a plurality of images recorded in advance, combining arecorded image with an image obtained on a real-time basis, and thelike.

THIRD APPLICATION EXAMPLE

In this example, image information and range information of the systemfor the light-section method as shown in the foregoing first to fourthembodiments are used in operation feedback control on various apparatusand robots.

For example, it is possible to automatically control an apparatusaccording to range information of an image in remote control operationin telemedicine.

Also, when an apparatus performs some operation for an object appearingin an image, a space in which the apparatus is allowed to operate can belimited so that the apparatus does not contact the object, for example.

In addition, when the system is incorporated into an autonomous robot orthe like, the robot can generate a map of the environment of a room bydetecting and storing information on an arrangement of furniture in theroom and the like. The map can be used as basic data information for therobot to move or work in the room.

FOURTH APPLICATION EXAMPLE

In this example, the system for the light-section method as shown in theforegoing first to fourth embodiments is used to recognize motion of anobject and gesture by analyzing range information of an object on a timeaxis.

The system as shown in the foregoing first to fourth embodiments canobtain three-dimensional range information in real time. Thus, byanalyzing change in the position of the object in a direction of a timeaxis and analyzing characteristics of the motion, the system canrecognize a pattern (gesture) of the human motion.

This can be used to enable user interface techniques using gesture, forexample. The user interface techniques can be used for user interfacesof personal computers (PC), games, robots, various AVs, IT apparatusesand the like.

In addition, the gesture recognition can be combined with informationobtained from a normal image to increase objects for recognition andefficiency of recognition.

FIFTH APPLICATION EXAMPLE

In this example, information on natural projections and depressions isused for object recognition, personal recognition, or security purposes.

For example, for a security purpose involving personal authentication,information on shape of the face of a person is registered in advancefor recognition, and when an unidentified person comes thereafter,information on projections and depressions of the face of the person iscompared with the information on the projections and depressions of thepreregistered person to determine coincidence of the information andthereby identify the person. In this case, since the sensor systemaccording to the foregoing first to fourth embodiments can obtain anormal image simultaneously, this personal authentication can becombined with personal authentication through image recognition as inthe first application example.

This personal authentication is not limited to the face, and can beperformed using various parts of the body.

Also, the sensor system according to the foregoing first to fourthembodiments can detect motion and recognize gesture through analysis inthe direction of the time axis, as illustrated in the fourth applicationexample. Thus, the gesture can be used for personal authentication.

Moreover, such authentication using projections and depressions can beused to identify not only persons but also ordinary objects.

Furthermore, instead of being used as accurate data for identificationand authentication as described above, information on projections anddepressions of parts of subjects can be analyzed as characteristics ofthe projections and depressions, for example characteristics of thetexture, and thus used as data for identification and authentication.

SIXTH APPLICATION EXAMPLE

In this example, the system for the light-section method as shown in theforegoing first to fourth embodiments is used to check the rear andoutside of a motor vehicle.

In checking the rear of a motor vehicle, for example, steering operationis performed while viewing a normal image provided by the sensor, and atthe same time, the system measures a distance to an obstacle in the rearand issues a warning when the vehicle comes near to a certain distanceto the obstacle.

Also, by setting a mark formed by projections and depressions on anordinary road or the like and reading the mark by the sensor systemincluded in an individual vehicle, it is possible to apply the sensorsystem to automatic control feedback of the motor vehicle or the like.For example, a system can be constructed in which a mark formed byprojections and depressions is set on a side portion of a road, themotor vehicle travels while reading the mark, and a warning is issuedwhen the motor vehicle is deviating from the road.

SEVENTH APPLICATION EXAMPLE

In this example, the system for the light-section method as shown in theforegoing first to fourth embodiments is used inside a motor vehicle.

For example, the three-dimensional measuring function is used todetermine presence of a person in a seat, age of a sitting person andthe like. Results of such detection can be fed back to issue a warningto wear a seat belt, adjust operating level of an air bag, for example.

Also, a gesture recognition function as in the fourth applicationexample is used so that a driver can control a device included withinthe vehicle by gesture without touching buttons or the like of thedevice.

EIGHTH APPLICATION EXAMPLE

In this example, the system for the light-section method as shown in theforegoing first to fourth embodiments is used for real-timethree-dimensional modeling.

Since the system according to the first embodiment can obtain a normalimage and perform three-dimensional measurement substantiallysimultaneously, the system is capable of three-dimensional modeling ofan object and texture mapping by cutting and pasting images. Theseprocesses can be carried out in real time.

NINTH APPLICATION EXAMPLE

As a ninth application example, the system for the light-section methodas shown in the foregoing first to fourth embodiments can be used forobject separation processing of MPEG4.

As described above, a solid-state image pickup apparatus according tothe present invention transmits an image signal obtained by an imagingpixel in a first signal transmitting path and a second signaltransmitting path of a signal line, and performs different signalprocessing operations by means of a first signal processing unit and asecond signal processing unit. Thus, normal image signal output andvarious other arithmetic processing, for example, can be performed bythe separate circuits.

In addition, a control method according to the present inventiontransmits an image signal obtained by an imaging pixel in a first signaltransmitting path and a second signal transmitting path of a signalline, and performs different signal processing operations in a firstsignal processing step and a second signal processing step. Thus, normalimage signal output and various other arithmetic processing, forexample, can be performed by different circuits.

Therefore, circuit elements required for each of the signal processingoperations can be arranged together outside pixels, so that aconfiguration within each of the pixels can be simplified and minimized.Also, it is possible to enhance the function of obtaining a normalactual image and the computational function for executing variousapplications by their respective independent circuit configurations. Itis thus possible to achieve smaller apparatus size, lower powerconsumption, lower cost, an increase in the number of pixels of anactual image (higher image quality) and the like.

1. A method of manufacturing a solid-state image pickup apparatuscomprising: forming a plurality of light receiving units; forming aplurality of photoelectric conversion units for converting lightreceived by said light receiving units into electric signals; providinga signal line having a plurality of signal transmission paths forextracting the electric signals converted by said plurality ofphotoelectric conversion units; providing a first signal processing unitfor processing the electric signals transmitted in a first signaltransmitting direction through said signal line; and providing a secondsignal processing unit for processing the electric signals transmittedin a second signal transmitting direction through said signal line;wherein the signal processing performed by said first signal processingunit and the signal processing performed by said second signalprocessing unit are different from each other.
 2. A method ofmanufacturing a solid-state image pickup apparatus as claimed in claim1, wherein said first signal processing unit performs image informationoutput processing, and said second signal processing unit performsarithmetic processing for executing a predetermined application usingoutput signals from said photoelectric conversion units.
 3. A method ofmanufacturing a solid-state image pickup apparatus as claimed in claim1, further comprising a switch for blocking a signal transmission pathin said second signal transmitting direction when said first signalprocessing unit performs the signal processing and blocking a signaltransmission path in said first signal transmitting direction when saidsecond signal processing unit performs the signal processing.
 4. Amethod of manufacturing a solid-state image pickup apparatus as claimedin claim 1, wherein the signal processing by said first signalprocessing unit and the signal processing by said second signalprocessing unit are selected by an external selecting signal.
 5. Amethod of manufacturing a solid-state image pickup apparatus as claimedin claim 1, wherein at a time of non-operation of said first signalprocessing unit and said second signal processing unit, part or all ofthe operation is stopped.
 6. A method of manufacturing a solid-stateimage pickup apparatus as claimed in claim 1, wherein order of scanningpixels is changed between the signal processing by said first signalprocessing unit and the signal processing by said second signalprocessing unit.
 7. A method of manufacturing a solid-state image pickupapparatus as claimed in claim 6, wherein as the order of scanning saidpixels, serial processing in a unit of one pixel or a block unit of asmall number of pixels and parallel processing using a plurality ofsignal lines are used.
 8. A method of manufacturing a solid-state imagepickup apparatus as claimed in claim 7, wherein serial processing in aunit of one pixel is performed at a time of image information outputprocessing, and parallel processing using a plurality of signal lines isperformed at a time of other arithmetic processing.
 9. A method ofmanufacturing a solid-state image pickup apparatus as claimed in claim1, wherein a number of pixels transmitted simultaneously to a singlesignal line is changed between the signal processing by said firstsignal processing unit and the signal processing by said second signalprocessing unit.
 10. A method of manufacturing a solid-state imagepickup apparatus as claimed in claim 9, wherein a signal of each pixelis separately transmitted to the signal line at a time of imageinformation output processing, and signals of a plurality of pixels aresimultaneously transmitted to the single signal line at a time of otherarithmetic processing.
 11. A method of manufacturing a solid-state imagepickup apparatus as claimed in claim 10, wherein a number of pixelscorresponding to a combination of color filters is used as a number ofpixels simultaneously transmitted to the single signal line at the timeof said arithmetic processing.
 12. A method of manufacturing asolid-state image pickup apparatus as claimed in claim 1, wherein eachof pixels includes: said light receiving unit; said photoelectricconversion unit; a transfer transistor for transferring a signal chargefrom said photoelectric conversion unit; a transfer selecting transistorfor selecting said transfer transistor; an amplifying transistor foramplifying the signal charge transferred by said transfer transistor andconverting the signal charge into an electric signal; a reset transistorfor resetting the signal charge supplied to said amplifying transistor,and a selecting transistor for selecting the pixel.
 13. A method ofmanufacturing a solid-state image pickup apparatus as claimed in claim12, wherein said pixels are arranged in a row direction and a columndirection in a two-dimensional matrix form within a pixel array.
 14. Amethod of manufacturing a solid-state image pickup apparatus as claimedin claim 13, wherein the selecting transistor of each of said pixels ineach column is selected collectively by a selecting line arranged in therow direction, and said signal line is arranged in the column direction.15. A method of manufacturing a solid-state image pickup apparatus asclaimed in claim 14, wherein a transfer selecting line for transmittinga signal charge transfer pulse to said transfer selecting transistor anda reset line for transmitting a reset pulse to said reset transistor arearranged in parallel with a row.
 16. A method of manufacturing asolid-state image pickup apparatus as claimed in claim 15, wherein saidtransfer selecting line is arranged in a certain row, and is also usedas said reset line arranged in a next row.
 17. A method of manufacturinga solid-state image pickup apparatus as claimed in claim 16, whereinsaid transfer selecting transistor has a drain electrode connected tosaid transfer selecting line arranged in parallel with the row and agate electrode connected to a transfer gate selecting line arranged inparallel with a column.
 18. A method of manufacturing a solid-stateimage pickup apparatus as claimed in claim 17, wherein as timing ofimage reading in a pixel arrangement in said two-dimensional matrixform, said transfer selecting line and said transfer gate selecting lineare activated simultaneously at a time of charge transfer, rind aftercompletion of the transfer, said transfer selecting line is inactivated,and then said transfer gate selecting line is inactivated.
 19. A methodof manufacturing a solid-state image pickup apparatus as claimed inclaim 18, wherein when column-parallel operation is performed in saidpixel arrangement, said transfer gate selecting line is activated at alltimes.
 20. A method of manufacturing a solid-state image pickupapparatus as claimed in claim 18, wherein in simultaneously readingpixels in a plurality of columns in said pixel arrangement, selectinglines in a plurality of rows are activated, and thereafter an activepulse is sequentially applied to selecting lines shared by transferselecting lines and reset lines in increasing order of selected rownumber.
 21. A method of manufacturing, a solid-state image pickupapparatus as claimed in claim 13, wherein a current mirror circuit isarranged in correspondence with a signal line arranged in the columndirection of said pixels.
 22. A method of manufacturing a solid-stateimage pickup apparatus as claimed in claim 21, wherein signal lines in aplurality of columns of said pixels are short-circuited for input to onecurrent mirror circuit.
 23. A method of manufacturing a solid-stateimage pickup apparatus as claimed in claim 21, wherein switches areprovided each for one or a plurality of columns of said pixels, a switchto be turned on at a time of signal reading is selected from theswitches, and thereby columns for input to individual current mirrorcircuits are selected.
 24. A method of manufacturing a solid-state imagepickup apparatus as claimed in claim 21, further including an analogmemory array disposed in the vicinity of said pixel array and having ananalog frame memory cell corresponding to each of said pixels, whereinsaid pixel and said analog frame memory cell corresponding to each otherare connected to each other by a signal path via said current mirrorcircuit.
 25. A method of manufacturing a solid-state image pickupapparatus as claimed in claim 24, wherein said pixel array and saidanalog memory array each have a separate scanner in the row directionand in the column direction.
 26. A method of manufacturing a solid-stateimage pickup apparatus as claimed in claim 25, wherein the scanners ofsaid pixel array and said analog memory array perform scanning insynchronism with each other.
 27. A method of manufacturing a solid-stateimage pickup apparatus as claimed in claim 25, further including meansfor simultaneously selecting a plurality of rows or a plurality ofcolumns within said pixel array, thereby merging and adding signals of aplurality of pixels together, and thus handling the plurality of pixelsas one light receiving pixel unit.
 28. A method of manufacturing asolid-state image pickup apparatus as claimed in claim 27, wherein insaid pixel array and said analog memory array, one frame memory cell ora plurality of frame memory cells are selectively made to correspond toone light receiving pixel unit.
 29. A method of manufacturing asolid-state image pickup apparatus as claimed in claim 28, wherein insaid analog memory array, when a plurality of frame memory cellscorrespond to one light receiving pixel unit, the plurality of memorycells are read simultaneously, whereby signals of the plurality ofmemory cells are merged and added together.
 30. A method ofmanufacturing a solid-state image pickup apparatus as claimed in claim29, wherein in said analog memory array, when a result of addition of acombination of two frames or more among a plurality of framescorresponding to one light receiving pixel unit is compared, memorycells corresponding to the frames are arranged in a matrix manner with asignal line in between, and cells on opposite pole sides with the signalline in between are selected as the combination of the frames at alltimes.
 31. A method of manufacturing a solid-state image pickupapparatus as claimed in claim 29, wherein in said pixel array and saidanalog memory array, selection of a plurality of rows by the scanner ofsaid pixel array at a time of selecting one light receiving pixel unitis synchronized with selection of a plurality of rows by the scanner ofsaid analog memory array at a time of selecting a unit of a plurality offrames.
 32. A method of manufacturing a solid-state image pickupapparatus as claimed in claim 24, wherein a current copier circuit isused as a circuit forming said analog memory array.
 33. A method ofmanufacturing a solid-state image pickup apparatus as claimed in claim32, wherein in said analog memory array, memory switch selecting linesand cell selecting lines for current copier scanning are arranged in arow direction.
 34. A method of manufacturing a solid-state image pickupapparatus as claimed in claim 24, wherein a signal of each said analogmemory is compared by using a comparator disposed for each column oreach plurality of columns, and differential calculation processing isthus performed.
 35. A method of manufacturing a solid-state image pickupapparatus as claimed in claim 34, wherein a switch is provided for eachcolumn between said analog memory array and said comparator, and theswitch is turned off at a time of memory writing and the switchnecessary for calculation is turned on at a time of memory reading. 36.A method of manufacturing a solid-state image pickup apparatus asclaimed in claim 35, wherein a plurality of column signal lines aremerged with each other in a stage preceding said comparator, and acolumn to be merged is selected by said switch.
 37. A method ofmanufacturing a solid-state image pickup apparatus as claimed, in claim34, wherein a chopper comparator is used as said comparator.
 38. Amethod of manufacturing a solid-state image pickup apparatus as claimedin claim 34, wherein a bias current is provided in a stage precedingsaid comparator, and performs arbitrary weighting at a time of readingthe signal from said analog memory.
 39. A method of manufacturing asolid-state image pickup apparatus as claimed in claim 38, wherein saidbias circuit uses a source follower circuit of a PMOS transistor biasedby a column-shared current mirror.
 40. A method of manufacturing asolid-state image pickup apparatus as claimed in claim 38, wherein inother than reading operation, a switch provided between said biascircuit and said analog memory is turned off, and thereby said biascircuit biases potential of the signal line in the stage preceding thecomparator to a power supply voltage.
 41. A method of manufacturing asolid-state image pickup apparatus as claimed in claim 34, wherein whensignals from said pixel array are read in column-parallel, andcalculation processing is performed in each column, calculation blocksare formed each by a plurality of columns, and said calculationprocessing is performed serially in said calculation blocks.
 42. Amethod of manufacturing a solid-state image pickup apparatus as claimedin claim 41, wherein said calculation block is selected by an addressline, and switches in stages preceding and succeeding a current mirrorunit, a memory switch line, a switch in a stage succeeding thecomparator, a line for selecting a load transistor of the signal line, acomparator initializing line, and an enable line of a data latch areoperated by the address line only when the block is selected.
 43. Amethod of manufacturing a solid-state image pickup apparatus as claimedin claim 1, wherein said first signal processing unit and said secondsignal processing unit are used for image signal output and rangemeasurement.
 44. A method of manufacturing a solid-state image pickupapparatus as claimed in claim 1, wherein said first signal processingunit and said second signal processing unit are used for image signaloutput and motion detection.
 45. A method of manufacturing a solid-stateimage pickup apparatus as claimed in claim 1, wherein said first signalprocessing unit and said second signal processing unit are used foranalog image signal output and digital image signal output.
 46. A methodof manufacturing a solid-state image pickup apparatus as claimed inclaim 1, wherein said first signal processing unit and said secondsignal processing unit are used for image signal output and image filterprocessing.
 47. A method of manufacturing a solid-state image pickupapparatus as claimed in claim 1, wherein said first signal processingunit and said second signal processing unit are used for image signaloutput and image smoothing processing.
 48. A method of manufacturing asolid-state image pickup apparatus as claimed in claim 1, wherein saidfirst signal processing unit and said second signal processing unit areused for image signal output and image edge detection.
 49. A method ofmanufacturing a solid-state image pickup apparatus as claimed in claim1, further comprising: forming a light source for irradiating a subject;and forming a scanning mirror for scanning light emitted from said lightsource and thus irradiating said subject with the light; wherein thelight reflected after irradiating said subject with the light by saidlight source and said scanning mirror is detected by said lightreceiving units, and arithmetic processing is performed by said secondsignal processing unit, whereby said subject is measured by alight-section method.
 50. A method of manufacturing a solid-state imagepickup apparatus as claimed in claim 49, wherein said light source emitsa stripe of light.
 51. A method of manufacturing a solid-state imagepickup apparatus as claimed in claim 49, wherein said scanning mirrorhas a mirror driving unit formed in a moving plate serving as a mirrorbody made of a semiconductor or other material.
 52. A method ofmanufacturing a solid-state image pickup apparatus as claimed in claim51, wherein said scanning mirror has a base for retaining said movingplate in such a manner as to allow rocking displacement of said movingplate, and said moving plate is attached to the base with a hingeportion between said moving plate and the base and is displaced in arocking manner by deformation of said hinge portion.
 53. A method ofmanufacturing a solid-state image pickup apparatus as claimed in claim52, wherein said scanning mirror further includes a magnetic circuit forapplying a magnetic field to said moving plate retained by said base,and said minor driving unit includes a driving coil for driving saidmoving plate by Lorentz force by being supplied with a current withinsaid magnetic field.
 54. A method of manufacturing a solid-state imagepickup apparatus as claimed in claim 53, wherein said minor driving unitfurther includes a detecting coil for detecting displacement of saidmoving plate within said magnetic field.
 55. A method of manufacturing asolid-state image pickup apparatus as claimed in claim 52, wherein saidmirror driving unit drives said moving plate using thermal expansion ofhinge material by passing a current through the hinge portion of saidmoving plate.
 56. A method of manufacturing a solid-state image pickupapparatus as claimed in claim 52, wherein said mirror driving unitdrives said moving plate using difference in thermal expansivity betweenlayered films forming the hinge portion by passing a current through thehinge portion of said moving plate.
 57. A method of manufacturing asolid-state image pickup apparatus as claimed in claim 52, wherein saidmirror driving unit has a galvanometric driving mode and a resonantdriving mode as driving mode.
 58. A method of manufacturing, asolid-state image pickup apparatus as claimed in claim 52, whereinpolyimide material is used in said hinge portion.
 59. A method ofmanufacturing, a solid-state image pickup apparatus as claimed in claim49, wherein said light source and said scanning mirror are disposed onan identical substrate.
 60. A method of manufacturing a solid-stateimage pickup apparatus as claimed in claim 49, further including meansfor controlling mirror vibration frequency, mirror swing angle, andstarting and stopping of operation of said scanning mirror by anexternal signal.
 61. A method of manufacturing a solid-state imagepickup apparatus as claimed in claim 49, wherein said light source is alaser light source, and has a laser hologram for converting laser lightinto a stripe of light.
 62. A method of manufacturing a solid-stateimage pickup apparatus as claimed in claim 61, wherein said laserhologram is disposed between said light source and said scanning mirror.63. A method of manufacturing a solid-state image pickup apparatus asclaimed in claim 61, wherein said laser hologram is disposed on asubject side of said scanning mirror.
 64. A method of manufacturing asolid-state image pickup apparatus as claimed in claim 49, whereinprocessing of extracting an image in a certain distance range from animage is performed on the basis of range information obtained by thearithmetic processing of said second signal processing unit.
 65. Amethod of manufacturing a solid-state image pickup apparatus as claimedin claim 49, wherein processing of replacing an image in a certaindistance range of an image with a different image is performed on thebasis of range information obtained by the arithmetic processing of saidsecond signal processing unit.
 66. A method of manufacturing asolid-state image pickup apparatus as claimed in claim 65, wherein saiddifferent image is manually generated by an operator.
 67. A method ofmanufacturing a solid-state image pickup apparatus as claimed in claim65, wherein said different image is inputted from a medium.
 68. A methodof manufacturing a solid-state image pickup apparatus as claimed inclaim 65, wherein said different image is picked up by said solid-stateimage pickup apparatus.
 69. A method of manufacturing a solid-stateimage pickup apparatus as claimed in claim 64, wherein a certain imageis extracted, and a subject is recognized by image recognition.
 70. Amethod of manufacturing a solid-state image pickup apparatus as claimedin claim 64, wherein a certain image is extracted, and a subject isrecognized by feature extraction using temporal analysis of positionalinformation of the subject.
 71. A method of manufacturing a solid-stateimage pickup apparatus as claimed in claim 64, wherein a certain imageis extracted, and a subject is recognized by image recognition andfeature extraction using temporal analysis of positional information ofthe subject.
 72. A method of manufacturing a solid-state image pickupapparatus as claimed in claim 64, wherein a certain image is extracted,a subject is recognized by image recognition and feature extractionusing temporal analysis of positional information of the subject, andthe subject is identified by comparing a result of the recognition witha result of recognition of another subject obtained by similarprocessing.